System for generating virtual clothing experiences

ABSTRACT

A system for generating a virtual clothing experience has a display for mounting with a wall, one or more digital cameras for capturing first images of the person standing in front of the display, an image processing module for synthesizing the first images and for generating a display image on the display that substantially appears, to the person, like a reflection of the person in a mirror positioned at the display. The cameras capture second images of a garment with the person; the module synthesizes the second images with the first images to generate the image that substantially appears, to the person, that the reflection wears the garment. A home version of the system may be formed with a home computer and a database storing garment images in cooperation with a manufacturer.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/115,005, filed May 24, 2011 which is a continuation of U.S.application Ser. No. 10/304,335, filed Nov. 26, 2002, now U.S. Pat. No.7,953,648, which claims priority to U.S. Provisional Application No.60/333,675 filed Nov. 26, 2001. Both of the aforementioned applicationsare incorporated herein by reference.

BACKGROUND OF THE INVENTION

Buying clothes can be a real chore. A person often travels to a store,and then tries on several articles of clothing in a changing room. Theperson often has to travel to other stores too in order to find desiredpurchases.

There is a need to simplify this process and to save time. It is,accordingly, one feature hereinbelow to provide a system that reduces oreliminates the need to physically try on clothing. Another featurehereinbelow is to provide a process to try on articles of clothing on athome, and in private. Other features are apparent within the descriptionthat follows.

SUMMARY OF THE INVENTION

In one aspect, a system includes a display that hangs on the wall. Thedisplay is for example a plasma or LCD display, or other flat paneldisplay; preferably, the display looks like a mirror that one would useto review and assess worn articles of clothing. An image processingmodule generates an image on the display based upon (a) a person infront of the display and (b) one or more articles of clothing (e.g., adress, shoe, shirt, pant). One or more digital cameras capture images ofthe person; those images are synthesized by the image processing moduleto generate the image on the display. The cameras may for exampleintegrate with the display as a modular unit.

In one aspect, the system self-identifies, and/or alternatively receivesa message identifying, one or more of the articles of clothing to betried on by the person. A database stores one or more images of thearticles, and the image processing module synthesizes the image on thedisplay as a representation of the person wearing the article(s). Inthis way, a person may select and try an article of clothing by viewingthe display and selecting which article to try on. If desired, and theperson likes the look of the article, then he may go to a changing roomto physically try on the article. The system thus saves repeated tripsto the changing room.

In one aspect, the message is generated by an RF signal communicated tothe system. In one aspect, the person selects the article of clothingfrom a list and presses a button indicating which of the articles shewould like to try on; pressing the button communicates the messagesignal to the system. In one aspect, that button is on a hanger or rackassociated with the article.

In another aspect, the system views an article of clothing and matchesthe article to an internal database to automatically generate the imagebased upon image recognition techniques. For example, a person stands infront of the display and holds an article up next to him; the systeminterprets this as a signal to search for the article within thedatabase. If the system identifies the article, it synthesizes thearticle onto the person, in the displayed image, so that the displayimage appears as if the article is worn on the person.

In one aspect, the invention also provides indication of article sizerelative to the person. Preferably, the system includes a plurality ofdigital cameras viewing the person; the positioning of the camerasprovides geometric information such that the system determines anapproximate size of the person (e.g., for a woman, a size 7, 10, 11 or12, tall, short, medium, slender, fat; and for a man with a 38 inchwaist and 30 inch inseam, or a 44 inch chest and 31 inch arm inseam).The system then determines which article size to show to the person,e.g., a size 7 dress or a “large” dress shirt.

In another aspect, the system receives a message indicating which sizethe person wishes to try on. Such a message may be communicated forexample by pressing of the button described above. In one aspect, if thesystem determines that the selected article size is inconsistent withthe person's actual size (e.g., as determined by the system), the systemmay display the image with a warning indicator. The indicator may takethe form of a red beacon in the display image in a location where thearticle does not fit the person. The indicator may also simply flash anLED indicating a “sizing problem,” suggesting that the person selectanother size (or let the system suggest or select another size).

In another aspect, a system provides for trying on articles of clothingat home. Preferably, like above, a display is hung in the person's hometo show articles selected by the user; the cameras are not necessarilyneeded. The display may also be the person's home computer. A personselects articles by codes or other means associated with articles ofclothing in a database. The database may be linked with a particularmanufacturer's clothing, for example Land's End clothing. The databasemay be linked directly to the display (computer display or an on-walldisplay) by the Internet. Preferably, a 3D representation of the personis loaded within system memory and that 3D representation is used whenselecting and trying on clothes. For example, the code of the articlemay correspond to a size “7” that is consistent with the 3Drepresentation of the person. Therefore, the system shows the image ofthe selected article properly overlaid with the person. If however theselected size is not appropriate for the person's 3D representation,then the system may so indicate, such as described above, and within thedisplayed image. Preferably, the 3D image includes, or couples with, anouter image “skin” of the person so that when the 3D representation andthe graphic of the article are synthesized, by the system, then theclothing appears naturally to the person; that is, she sees herself (andwith skin and facial features) wearing the article in a way she is usedto seeing articles on her body, except for the fact that the image isnot a “reflection” but rather a synthesized image of her in the desiredarticle.

In one aspect, the person selects the article for purchase automaticallythrough the system. Preferably, the system integrates seamlessly withone or more manufacturers or sales stores that will process the order.

The system thus provides for virtually trying on various articles ofclothing without actually putting them on.

U.S. Pat. No. 5,850,222 provides useful background for the invention andis hereby incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a side-view of one system in use within a store;

FIG. 1A shows a front view of the display of FIG. 1;

FIG. 2 shows a flowchart illustrating one process of a system such asshown in FIG. 1 and FIG. 1A; and

FIG. 3 shows one system in use within a person's home.

DETAILED DESCRIPTION OF THE DRAWINGS

FIG. 1 and FIG. 1A show a system 10 for generating a virtual clothingexperience. A display 12 provides an electronic image 14 of a person 16standing in front of display 12. A plurality of cameras 18 captureimages of the person 16; each of the cameras 18 has a field of view 20,as shown. Frames of images from cameras 18 are communicated to an imageprocessing module 22 to synthesize the frames into image 14. Asdescribed below, image 14 may be substantially similar to the normalimage expected by person 16 if person would view a reflection of herimage from a mirror at the location of display 12. That is, system 10may substantially reproduce the normal “reflective” image 14 of person16, including normal skin and facial features; however, system 10 mayalso synthesize an image 40′ of an article of clothing 40 within image14 so that it appears as if the person is wearing article 40 normally,but without physically putting clothing 40 on person 16. Cameras 18 arepreferably color digital cameras; and display 12 preferably displayscolor synthesized images from module 22 in forming image 14.

Preferably, cameras 18 are arranged with geometrical coordinates suchthat combination of frames from multiple cameras 18 provides anestimation of the size of person 16. By way of example, camera 18 b and18 c define edges 30 (represented by one or more pixels of cameras 18 b,18 c) of person 16; one camera 18 a may also include auto-focuscapability to define a distance D between display 12 and person 16. Incombination, module 22 estimates the size of person 16 by thetrigonometry associated with edges 30 and distance D. Other techniquesmay be used to estimate the person's size without departing from thescope hereof. By way of example, center image lines 32 may also define aheight and width of person 16. System 10 may be made with a singlecamera 28, preferably a center camera 18 a(1) to obtain a frontal viewof person 16; however multiple cameras 18 enhance the image processingand realism of image 14.

Module 22 also preferably includes body form heuristics logic 34 thatassists in estimating the size of person 16. Logic 34 may for examplemodify an estimation of a person's size, defined for example bygeometries 30, 32, to account for body forms such as thin, fat, largewaist, short upper torso, etc. Logic 34 may also include neural networklogic to “learn” to help system 10 better identify body shapes andsizes.

In one embodiment, module 22 estimates the clothing size (e.g., small,medium, large, extra large, size 6, 7, etc.) of person 16 based on thesize estimations described above and according to manufacturing orindustry conventions for clothing sizing. In one embodiment, database 42stores images of clothing such as clothing 40. When appropriate, system10 synthesizes the images within database 42 into image 14, togetherwith a synthetic image based upon frames of data from cameras 18, todisplay article 40′ on image 14 as if person 16 had actually tried onclothing 40. An image 40′ within database 42 may be directly scaled andoverlaid on the image of person 16 to define image 14, or images withindatabase 42 may be synthesized to provide a more natural-looking image14.

System 10 either determines which size of clothing 40′ to place ontoimage 14, or person 16 selects the clothing size. Person 16 may select asize by pressing a button 44 to identify article 40; button 44 may beanywhere within the store as a matter of design choice, such as with thehanger 48 holding clothing 40 on a rack of the store. Further, button 44preferably couples with an RF transmitter to communicate the size to anantenna 46 of module 22 as RF signals 50. System 10 may inform the userof the size (selected manually by person 16 or automatically) in adisplay segment 12 b of display 12. Segment 12 b may alternatively be aseparate display such as an LED display.

In one embodiment, database 42 stores images therein of articles ofclothing, such as clothing 40, and as worn by a user of various bodysizes. That is, images within database 42 may be “real” digital picturesof articles of clothing on various persons with different body sizes,and in different positions. In operation, therefore, system 10 maydetermine the body size of person 16 and then more easily overlay animage of article 40 with an appropriate image from database 42. Forexample, if system 10 determines that person 16 is a size 7, it mayacquire an image of article 40 in the database and on a previouslyacquired image of a person with a size 7 body, and in a position similarto person 16 in front of display 12. Module 22 then synthesizes theimages of person 16, captured by cameras 18, with the image of clothing40 on a similarly sized person (and preferably in a similar position asperson 16). Image processing module 22 may execute one or morecross-correlations of images within database 42 with images from cameras18 to determine which position and body size provide the best “match”(similar to match filtering, known in the art).

In the even that system 10 determines that a clothing size does not“fit” the body form of person 16, it may indicate a problem by a displaysegment 12 c. Segment 12 c may alternatively be a LED or otherindicator. Segment 12 c may show “red” illuminating the problem toperson 16; segment 12 c may be part of the display of article 40, asshown in FIG. 1A, illustrating where the article does not fit person 16at that body location.

System 10 may use image recognition techniques to automatically matchclothing 40 to images within database 42. Alternatively, person 16 mayselect the article such as by pressing button 44, so that system 10expressly knows which clothing images to synthesize within image 14.

Cameras 18 may be integrated with display 12 as shown, to provide amodular system, or they may be separate from display 12, as a matter ofdesign choice. In one embodiment, at least one camera 18 a views“through” display 12. Alternatively, one or more cameras 18 b, 18 c maybe adjacent to display 12, as shown. A camera such as camera 18 a(1) mayalso view person 16 through a hole within display 12, without departingfrom the scope hereof; such a hole has little overall impact on image 14viewed by person 16. Cameras 18 typically connect with module 22 bysignal lines 60, which may include power, video and/or network cabling.

Display 12 is for example a LCD display, flat panel display, or plasmadisplay, that show images at its display face 12 a and under control ofmodule 22. Module 22 may transmit display information to display 12 viasignal line 61, for example.

As an alternative to database 42, system 10 may capture an image ofarticle 40 in “real time” and synthesize that image with the image ofperson 16 as image 14. That is, cameras 18 capture images of article 40,particularly when person 16 holds up article 40 in front of him; system10 of one embodiment uses this information to generate image 14 (withimage 40′ of article 40).

FIG. 2 shows a flowchart 100 illustrating one operation with softwaresteps for one system 10. After start, person 16 stands in front of thedisplay 12, in step 102. At step 104, digital image frames of person 16are captured by cameras 18 with display 12. At step 106, those framesare synthesized to form an image 14 on display 12 that is similar to howperson 16 would appear, from the perspective of person 16, in a mirroredreflection from a minor positioned at display 12. In step 108, person 16selects a garment 40 to try on. If images of the garment 40 are notavailable in database 42, decided at step 110, then images of garment 40are acquired, in step 112. Those acquired images are integrated (asimage 40′) with image 14 of person 16, in step 114, so that it appears,to person, 16 that garment 40 is being worn when viewing display 12. Ifimages of garment 40 are available, in database 42, decided at step 110,then database images of garment 40 are located, at step 116, and thenintegrated and/or synthesized (as image 40′) with image 14 of person 16,as in step 114.

FIG. 3 shows one system 200 for generating a virtual clothingexperience. A person 202 uses system 200 to try on an article ofclothing 204 (clothing 204 is an image, not actual clothing) at a homecomputer 206. Though not required, computer 206 may connect with adatabase 208 through the Internet 210. Database 208 may be controlled bya clothing store or manufacturer; and thus system 200 may be used topurchase clothing from the store or manufacturer. System 200 may also beused to try on articles of clothing 204 such as found in a direct mailcatalog 212, where images of clothing 204 are made available and aperson selects a clothing identifier to “try on” the selected garment byviewing images of himself at computer 206.

Computer 206 is specially programmed according to the teachings herein.Preferably, a body form matrix is stored within computer 206. By way ofexample, a body form matrix may take the form of an avatar, known in theart. The body form matrix provides sufficient information about person202 at computer 206 so as to appropriately size that person forclothing. By way of example, the body matrix may define the person as asize 6, slender. Preferably, skin and facial feature images of theperson are also stored within computer 206, so that computer 206synthesizes the body matrix with the skin and facial features togenerate an image 214 that appears substantially “real” to person 202 atcomputer 206. Image 214 includes the image of clothing 204′—which may beidentical to image 204 in catalogue 212; however, preferably image 204′is synthesized within computer 206 so that image 214 is in the position(e.g., standing, sitting, back view, front view) desired by person 202.Such synthesis seeks to make image 214 as “real” as possible so thatperson 202 has good feel for how clothing 204 would appear on person202. Computer 206 may be programmed to show multiple views of image 214,such as showing person 202 in different positions, to “model” clothing204 within computer 206.

In one embodiment, computer 206 and database 208 cooperate to provideimage 214. In particular, the image segment 204′ may derive fromdatabase 208; database 214 may in fact store the body matrix incooperation with person 202 such as by a confidential account to buyclothes from the store or manufacturer.

Image 214 may include information showing person 202 that clothing 204does not properly fit, such as by illuminating one or more pixels of thecomputer screen of computer 206 with “red” indicating a problem (e.g.,similar to segment 12 c, FIG. 1A). Indeed, a manufacturer or storesupporting database 208 may use the avatar or 3D body matrix topredetermine how a particular garment 204 will fit a person of similarsize. Specifically, each image of catalog 212 or database 208 mayinclude “fit” information linked to a particular body matrix. Forexample, if a body matrix of size large 10 attempts to wear a garment204 of slender 8, system 200 would indicate that a sizing problemexists, such as through colored pixels warning of the misfit. System 200may “suggest” the correct size, preferably in cooperation with themanufacturer or store of database 208.

Since certain changes may be made in the above methods and systemswithout departing from the scope hereof, it is intended that all mattercontained in the above description or shown in the accompanying drawingbe interpreted as illustrative and not in a limiting sense. It is alsoto be understood that the following claims are to cover all generic andspecific features described herein, and all statements of the scope ofthe invention which, as a matter of language, might be said to fallthere between.

What is claimed is:
 1. A virtual clothing experience system, comprising:an electronic display; cameras for capturing digital frames of a personin front of the electronic display; and computer processor forprocessing the digital frames with at least one garment image togenerate a real-time electronic image on the display that appears, tothe person, like a reflection of the person from a mirror positioned atthe display and wearing the garment.
 2. A system of claim 1, thecomputer processor synthesizing the digital frames with the garmentimage to generate the real-time electronic image, with the garment imagesuperimposed on the person.
 3. A system of claim 1, the processorgenerating an indication, on the electronic display, of improperclothing fit within the real-time electronic image.
 4. A system of claim3, said indication comprising colors within the real time electronicimage and co-located with a section of the electronic image associatedwith the improper clothing fit.
 5. A system of claim 1, the processorsizing the person through geometries associated with orientation of thedigital cameras capturing the digital images.
 6. A system of claim 5,wherein the digital cameras are arranged in a fixed orientation to theelectronic display.
 7. A system of claim 6, the digital cameras beingintegrated with the electronic display.
 8. A system of claim 1, whereinthe digital cameras image the person to generate the digital frames, andalso image a garment adjacent to the person to generate the garmentimage.
 9. A method of claim 8, the processor automatically determiningclothing size based on images of a garment adjacent to the person.
 10. Asystem of claim 1, the processor comprising means for electronicallysearching a database for the garment image.
 11. A system of claim 10,the processor synthesizing the garment image, found in the database, togenerate the real-time electronic image.